Variance-Based Exploration for Learning Model Predictive Control

The combination of model predictive control (MPC) and learning methods has been gaining increasing attention as a tool to systems that may be difficult model. Using MPC function approximator in reinforcement (RL) is one approach reduce the reliance on accurate models. RL dependent exploration learn, currently, simple heuristics based random perturbations are most common. This paper considers variance-based geared towards using approximator. We propose use non-probabilistic measure uncertainty value value-based methods. Uncertainty measured by variance estimate inverse distance weighting (IDW). IDW framework computationally cheap evaluate therefore well-suited an online setting, already sampled state transitions rewards. gradient then used perturb policy parameters direction where increasing. proposed method verified two simulation examples, considering both linear nonlinear system dynamics, compared standard perturbations.